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SPECTRALLY CONSTRAINED IMPULSE SHORTENING FILTER FOR A 
DISCRETE MULTI-TONE RECEIVER 
5 Background 

The invention relates to time-domain equalization in 
a discrete multi-tone (DMT) receiver. 

Conventional single carrier modulation techniques 
translate data bits for transmission through a communication 
10 channel by varying the amplitude and/or phase of a single 
sinusoidal carrier. By contrast, DMT, which is also 
referred to as Orthogonal Frequency Division Multiplexing 
(OFDM) or Multicarrier Modulation (MCM) , employs a large 
number of sinusoidal subcarriers, e.g., 128 or 256 
15 subcarriers. The available bandwidth of the communication 
channel is divided into subchannels and each subchannel 
communicates a part of the data. A DMT system may employ 
quadrature amplitude modulation (QAM) for each of the 
subcarriers . 

2 0 OFDM-based systems transmit blocks of information 

bits. The time required to transmit one such block is 
called the symbol period. The time domain waveform that 
corresponds to one such block of bits is called a symbol. 
Intersymbol interference (ISI) arises from the 

2 5 characteristics of practical communication channels and 
limits the rate at which information can be transmitted 
through them. Specifically, communication channels 
typically have an Effective Discrete-Time Impulse Response 
(EDIR) that is greater than one sample time in length, which 

30 causes ISI. ISI is a well-known phenomenon in single- 
carrier communication systems and there are many techniques 
for reducing it. The process of such ISI reduction is 
called equalization. ISI is discussed, for example, in 
Proakis, Digital Communications, McGraw Hill, 2nd Edition, 

35 1989. 



Equalization in OFDM-based systems is achieved by a 
two stage process. First, at the transmitter, an end- 
portion of each symbol is affixed to the beginning of the 
symbol to form what is called a Cyclic Prefix (CP) . A 
5 cyclic prefix that is greater than the EDIR of the channel 
prevents one symbol from interfering with another. 
Furthermore, it also facilitates a simple method of 
neutralizing the time-domain spread of each symbol forced by 
the channel. This is achieved by the use of a simple 

10 frequency domain process in the receiver which requires one 
multiplication operation for each used subcarrier of the 
OFDM system. 

The use of a Cyclic Prefix to reduce ISI is 
discussed, for example, in: Cimini, "Analysis and Simulation 

15 of a Digital Mobile Channel using Orthogonal Frequency 

Division Multiplexing," IEEE Transactions on communications, 
pp 665-675 July 1985; Chow, "A Discrete Mult i -Tone 
Transceiver System for HDSL applications," IEEE Journal on 
Selected Areas of Communications, 9 (6) : 895-908 , August 1991; 

20 "DMT Group VDSL PMD Draft Standard Proposal," Technical 
Report, T1E1.4/96-329R2, ANSI 1997. 

Another problem arising in conventional DMT systems 
is noise bleeding, which is the phenomenon of noise in one 
frequency band interfering with a signal whose subcarrier is 

25 in another frequency band. Noise bleeding is caused, in 
general, by the Discrete Fourier Transform (DFT) operation 
at the receiver. Noise bleeding is discussed in, for 
example, Worthen et. al . , "Simulation of VDSL Test Loops," 
Technical Report T1E1 . 4/97-288 , ANSI 1997. 

30 In a perfectly synchronized DMT system, the signal 

in one frequency band does not interfere with a signal whose 
subcarrier is in another frequency band. However, noise 
from one band may interfere with other less noisy bands and 
render them unusable. Techniques for dealing with noise- 



bleeding include wavelet-based solutions. However, wavelet- 
based solutions are, in general, computationally intensive. 

Summary 

The invention provides a spectrally constrained 
5 impulse shortening filter (SCISF) for use, for example, in 
DMT systems. The SCISF serves two primary functions. 

First, the SCISF reduces intersymbol interference 
(ISI) by reducing the length of the effective discrete-time 
impulse response (EDIR) of the communication channel. 

10 Conventional impulse shortening filters may have deep nulls 
in their frequency response. By contrast, the SCISF has a 
filter characteristic that is essentially free from 
undesired nulls that may attenuate or completely eliminate 
certain subcarriers. 

15 Second, the SCISF reduces noise bleeding between 

subchannels by attenuating noisy channels in a manner that 
does not reduce the signal to noise ratio (SNR) in these 
channels, but reduces the noise power that may appear in the 
sidelobes of adjacent subchannels. The SCISF accomplishes 

2 0 these functions by applying a frequency constraint to the 
signal based on a target spectral response. 

In one general aspect, the invention features 
equalizing a channel in a multiple carrier communication 
system. The system includes a spectrally constrained 

2 5 impulse shortening filter. Received noise power spectral 

density is measured. A target spectral response is computed 
based on the measured noise power. A frequency response for 
the spectrally constrained impulse shortening filter is 
selected based on the target spectral response. The 

30 communication signal is filtered with the spectrally 
constrained impulse shortening filter. 

Embodiments may include one or more of the following 
features. The noise power spectral density may be measured 
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at th output of the discrete Fourier transform. The 
spectrally constrained impulse shortening filter may be a 
time domain digital filter. 

In another aspect, the invention features equalizing 
5 a channel in a multiple carrier communication system. The 
channel has an impulse response and is configured to receive 
a signal having a cyclic prefix. A target spectral response 
is computed. The impulse response of the channel is 
shortened so that a significant part of the energy of the 

10 impulse response is confined to a region that is shorter 
than a target length, and the signal is filtered based on 
the target spectral response. 

The target length may be a length of the cyclic 
prefix. The target spectral response may be computed from 

15 measured noise power density, which may be measured at the 
output of a discrete Fourier transform. For example, the 
target spectral response may be the inverse of the measured 
noise power spectral density. The filtering step may be 
performed with a filter having a frequency response selected 

2 0 to match the target spectral response. The shortening of 
the impulse response and/or the filtering may be performed 
by a time domain digital filter. 

In another general aspect, the invention features 
selecting an impulse response for a spectrally constrained 

2 5 impulse shortening filter in a multiple carrier 

communication system. Received noise power spectral density 
is measured. A cost function is computed using the noise 
power. The cost function is dependent on the impulse 
response. The dimensionality of a space over which the cost 

3 0 function is defined is reduced and the cost function is 

minimized. The noise power spectral density may be measured 
at the output of a discrete Fourier transform. The cost 
function may be used to compute coefficients for the 
spectrally constrained impulse shortening filter. 



In another general aspect, the invention features a 
spectrally constrained impulse shortening filter for a 
multiple carrier communication system. The system includes 
a channel that has an impulse response. A digital filter 
structure receives the signal and apply a frequency 
characteristic to the signal. The frequency characteristic 
is determined by filter coefficients. Filter coefficients 
are selected to shorten the impulse response of the channel 
so that a significant part of the energy of the impulse 
response is confined to a region that is shorter than a 
target length and to apply a frequency characteristic to the 
signal based on a target spectral response. The selected 
filter coefficients are input to the taps of the filter. 

In another general aspect, the invention features a 
receiver for receiving a multiple carrier signal from a 
communication channel having an impulse response. An 
analog-to-digital converter receives the signal from the 
communication channel. A spectrally constrained impulse 
shortening filter receives the signal from the analog- to- 
digital converter and shortens the impulse response of the 
channel so that a significant part of an energy of the 
impulse response is confined to a region that is shorter 
than a target length. The filter also applies a frequency 
characteristic to the signal based on a target spectral 
response. A discrete Fourier transform receives the output 
of the spectrally constrained impulse shortening filter and 
a decoder receives outputs of the discrete Fourier 
transform. 

In another aspect, the invention features a modem in 
which an encoder receives digital data and outputs a 
constellation of complex values. An inverse discrete 
Fourier transform receives the constellation from the 
encoder. A digital-to-analog converter is connected to the 
inverse discrete Fourier transform and outputs a signal to a 



communication channel. An analog-to-digital converter 
receives the signal from the communication channel. A 
spectrally constrained impulse shortening filter shortens an 
impulse response of the channel so that a significant part 
5 of an energy of the impulse response is confined to a region 
that is shorter than a target length. The signal is 
filtered based on a target spectral response. A discrete 
Fourier transform connected to the filter and a decoder is 
connected to the discrete Fourier transform and outputs 

10 digital data. 

In another aspect, the invention features software 
for causing a processor in a communication system to perform 
the following operations: measure received noise power 
spectral density; and compute a target spectral response 

15 based on the measured noise power. The software may also 
include instructions for causing a processor to compute 
filter coefficients based on the target spectral response. 

In another aspect, the invention features software 
for causing a processor in a communication system to measure 

2 0 received noise power spectral density. A cost function is 
computed using the noise power. The cost function is 
dependent on an impulse response of a spectrally constrained 
impulse shortening filter. The dimensionality of a space 
over which the cost function is defined is reduced, and the 

25 cost function is minimized. 

The techniques described here are not limited to any 
particular hardware or software configuration. They may 
find applicability in any computing or processing 
environment that may be used for a communication system. 

30 The techniques may be implemented in hardware or software, 

or a combination of the two. Preferably, the techniques are 
implemented in computer programs executing on a digital 
signal processor that includes a processor and a storage 
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medium readable by the processor (including volatile and 
non-volatile memory) • 

Other features and advantages will be apparent from 
the following description, including the drawings and the 
5 claims . 



Brief Description of the Drawings 
Fig. 1 is a block diagram of a discrete multi-tone 
communication system. 

Fig. 2 is a plot of effective discrete- time impulse 
10 response (EDIR) of a communication channel including 
transmit and receive filters. 

Fig. 3 is a plot of the shortened EDIR due to a 
spectrally constrained impulse shortening filter (SCISF) . 
Fig. 4 is a block diagram of a SCISF. 
15 Fig. 5 is a plot of transmit signal power, signal 

power at SCISF input and noise power at SCISF input. 

Fig. 6 is a plot of signal and noise power at the 
output of the SCISF. 

Fig. 7 shows the filter response of a discrete 
20 Fourier transform for one frequency band. 

Fig. 8 is a plot of the target spectral response of 
the SCISF, G d (w) , versus the actual frequency response, 
G(cj) . 

Fig. 9 is a plot of signal-to-noise ratio at the 
25 output of the SCISF, the output of the DFT with the SCISF 
and the output of the DFT without the SCISF. 



Description 

As shown in Fig. 1, a discrete multi-tone (DMT) 
communication system 10 has a transmitter 12 and a receiver 
30 14. The transmitter 12 accepts an input data bit stream 
which passes through a constellation encoder 20. The 
encoder 2 0 divides the serial input bit stream into blocks 
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of data. These blocks of data are further subdivided into 
smaller blocks corresponding to subchannels. Each of these 
smaller blocks are used to compute a complex value 
representing a constellation point. Each constellation 
5 point corresponds to a subsymbol . The subsymbols are then 
output by the encoder. Taken together, the subsymbols 
constitute a symbol. 

The subsymbols are supplied to an inverse discrete 
Fourier transform (IDFT) 30, which may be implemented, for 

10 example, in a digital signal processor. The IDFT 30 outputs 
N time samples of a symbol . The time samples are processed 
by a parallel to serial converter 40 to form a single stream 
of time samples. 

Following the parallel to serial converter 40, a 

15 prefixer 50 adds a cyclic prefix to the beginning of each 
symbol to reduce intersymbol interference (ISI) . 
Alternatively, the cyclic prefix may be added in the 
parallel to serial converter. After the cyclic prefix is 
added, the resulting signal passes through a digital-to- 

20 analog (D/A) converter 60 for transmission to the receiver 
14 through a communication channel 70. An analog transmit 
filter 65 may be included following the D/A converter to 
band limit the transmitted signal. 

At the receiver 14, the signal passes through an 

25 analog-to-digital (A/D) converter 75 and then through an 
impulse shortening filter 90. An analog receive filter 85 
may be included prior to the A/D converter in order to band 
limit the received signal. A prefix stripper 100 strips the 
cyclic prefixes from the resulting symbols and a serial to 

30 parallel converter 110 divides the stream of time samples 
into parallel signal paths that form the inputs to a 
discrete Fourier transform (DFT) 120. The DFT 120 converts 
the time samples into subsymbols. A decoder 130 converts 
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the subsymbols into a data bits and outputs the resulting 
data. 

As discussed above, a cyclic prefix is added to each 
symbol prior to transmission through the communication 
5 channel to reduce the effects of ISI. The cyclic prefix is 
formed by copying the last v time samples from the end of a 
symbol and placing them at the beginning of the symbol. To 
eliminate ISI, the length of the cyclic prefix, v, is chosen 
to be longer than the effective discrete-time impulse 

10 response (EDIR) of the channel. However, because the cyclic 
prefix constitutes redundant data, increasing the length of 
the cyclic prefix reduces the efficiency of the communica- 
tion system. For example, in a system having N time samples 
per symbol and a cyclic prefix of v time samples, the 

15 efficiency of the system will be reduced by N/(N+v) . To 

maximize efficiency it is necessary either to minimize v or 
to maximize N. However, increasing N increases the 
complexity, latency and computational requirements of the 
system and at some point becomes impractical. Accordingly, 

20 it is desirable to minimize v. 

An impulse shortening filter having an impulse 
response, g(n), may be employed in the receiver to minimize 
the length of the cyclic prefix by decreasing the EDIR, 
g (n)*h(n) , of the effective communication channel, which 

25 includes the transmit and receive filters, the impulse 
shortening filter and the physical transmission channel. 
The use of an impulse shortening filter is referred to as 
time domain equalization. Decreasing the EDIR allows a 
shorter cyclic prefix to be used without increasing ISI. 

3 0 Fig. 2 is a plot of the EDIR for a DMT test 

configuration having a communication channel that is 4500 
feet in length and operates at a sampling frequency of 
11.04 MHz (test loop 4, as described in "Very-high Speed 
Digital Subscriber Lines: System Requirements," Technical 



Report T1E1.4/97-131R1 , ANSI 1998) . Th EDIR includes the 
effects of a transmit filter, the communication channel and 
a receive filter. Fig. 3 shows the impulse response as 
shortened by the addition of an impulse shortening filter. 
5 The impulse shortening filter is selected so that a 

significant part of the energy of the joint impulse response 
of the filter and the effective communication channel, 
g(n)*h(n) , is confined to a region that is shorter in length 
than the length of the cyclic prefix. Some prior algorithms 

10 for computing g(n) considered only shortening the EDIR and 
did not consider the spectral characteristics of the 
resulting impulse shortening filter. Such filters often had 
deep nulls in some frequency bands, which rendered some of 
the corresponding subchannels useless. 

15 Since increasing the length of the cyclic prefix 

reduces system efficiency, the receiver may dynamically 
compute an optimal length for the cyclic prefix and may send 
that information to the transmitter. For example, the 
receiver may compute a set of impulse responses for the 

20 impulse shortening filter based on a set of predetermined 
cyclic prefix lengths. The receiver then computes the 
system throughput for each particular cyclic prefix length. 
The length that maximizes system throughput is selected and 
the result of that selection is communicated to the 

25 transmitter. The transmitter then operates using the 
selected cyclic prefix length. 

To avoid the possible attenuation of frequency 
bands, the spectral response of the impulse shortening 
filter is further required to have a spectral response that 

30 |g(cj)|, meets a specified spectral constraint. A spectral 
constraint of the form | G(u)H(o)) | >t, where t is a threshold, 
is sufficient to avoid nulls in the frequency response of 
the impulse shortening filter. However, it is possible to 
compute a spectral constraint or target spectral response, 
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I G^a*; I, that provides additional performance improvements, 
such as reducing noise bleeding between subchannels. A 
spectrally constrained impulse filter is configured to have 
a spectral response that approximates the target spectral 
5 response . 

As shown in Fig. 4, the spectrally constrained 
impulse shortening filter (SCISF) 90 may be implemented as a 
time domain digital filter, which has a digital filter 
structure 210 with a number of taps 220 or filter 

10 coefficient inputs for adjusting the filter response. The 
coefficients may be computed and supplied to the taps by a 
digital signal processor (DSP) . Alternatively, the SCISF 
may be implemented entirely in software, i.e., within a DSP. 
A desired spectral response may be applied to a 

15 received signal using a filter that is separate from the 

impulse shortening or time domain equalization (TEQ) filter. 
For example, an analog filter may be placed prior to the A/D 
converter. However, the adjustability of such a filter 
would be limited. As a further example, a digital filter 

20 could be added prior to the TEQ filter. Both of these 

configurations suffer the disadvantage that the TEQ filter 
may distort the desired spectral characteristics of the 
added filter. A filter also might be positioned after the 
TEQ filter, which would reduce noise bleeding, but might 

25 reduce the impulse shortening provided by the TEQ filter. 
Accordingly, the SCISF integrates the TEQ (i.e., impulse 
shortening) function with the desired spectral response in a 
single filter. 

In summary, the filter characteristic, g(n), of the 

3 0 SCISF satisfies two conditions. First, the effective length 
of the convolution of the filter characteristic with the 
impulse response of the communication channel, g(n)*h(n) , is 
less than a target length. Second, the cost function (error 
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function) between the target spectral response, G d (w) , and 
the actual filter spectral response, G(o)), is minimized. 

The target spectral response is an ideal filter 
characteristic that is selected to maximize the data bit 
5 throughput in the subchannels of a DMT system by reducing 
the impact of noise. There are many sources for noise in a 
DMT communication system, such as near-end cross-talk 
(NEXT) , radio frequency interference (RFI) and noise 
generated in the communication channel (white noise) . As 

10 shown in Figs. 5 and 6, the noise spectral density is 
generally not uniform across the frequency band of the 
communication system. This non- uniformity contributes to 
the problem of noise bleeding, in which noise in one 
frequency band interfering with a signal in another 

15 frequency band. 

In general, noise bleeding is caused by sidelobes of 
filters in a DFT. Fig. 7 shows the filter response of a DFT 
for one frequency band or bin (i.e., bin 128) . The first 
sidelobes (250) are only 13 dB below the main lobe (260) . 

20 Therefore, noise located outside of bin 128, but within the 
first sidelobe of bin 128, i.e., approximately mid-way 
between bins 126 and 127, would appear in bin 128 with an 
attenuation of only 13 dB. Consequently, noisy subchannels 
in a DMT system may degrade the performance of non-noisy 

25 subchannels. 

The target spectral response is essentially a 
spectral constraint that attenuates noisy channels more than 
non-noisy channels. The signal and noise in the noisy 
channels are reduced equally, so the attenuation does not 

30 affect the signal-to-noise ratio in these channels. 

However, because the absolute noise level in the noisy 
channels is reduced, there is less noise available to enter 
the sidelobes of the non-noisy channels. Hence, the ncise 
bleeding problem is minimized. 
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To determine the target spectral response, the noise 
power spectral density (noise PSD) at the receiver must be 
known. The noise PSD may be determined, for example, by 
performing a periodigram on received data. This measurement 
is more complicated if a transmitter is transmitting, since 
the transmitted signal must be separated from the noise 
measurement. The noise PSD is determined by: (i) slicing 
the received constellation of subcarriers after the DFT to 
determine the nearest valid constellation point; (ii) 
determining an error signal based on the difference between 
the received constellation point and the valid constellation 
point; (iii) performing an IDFT on the error signal; and 
(iv) generating a periodigram (with windowing) from the 
error signals. The noise PSD may then be determined from 
the periodigram. 

An example of a noise PSD characteristic for a DMT 
communication system is shown in Fig. 5 (test loop 4, as 
described in "Very-high Speed Digital Subscriber Lines: 
System Requirements, " Technical Report T1E1 . 4 /97-131R1 , ANSI 
1998) . The transmit signal power is measured at the output 
of the transmit filter in the transmitter. The signal and 
noise PSD plots shown in Fig. 5 are measured at the input of 
the A/D converter in the receiver, which is prior to the 
SCISF. 

Measured noise PSD is used by a digital signal 
processor (DSP) to compute a target spectral response, 
G d (o)) , using the algorithm described below. Alternatively, 
the inverse of the noise PSD may be used as an approximation 
for the target spectral response. A spectral response, 
G(u), is then determined for the SCISF that minimizes the 
error between the spectral response of the SCISF and the 
target spectral response. A set of filter coefficients may 
then be generated to configure the SCISF to the determined 
characteristic. These calculations may be done periodically 
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to adjust the performance of the communication system. 
Frequency domain equalization coefficients and symbol 
synchronization may also be adjusted based on these 
calculations. 

Fig. 8 is a plot of the target spectral response of 
the SCISF, G d (w) , versus the actual frequency response, 
G(u) . The difference between the responses is only a few 
dB. Fig. 6 shows the signal and noise PSD at the output of 
the SCISF. 

Fig. 9 shows the dramatic effect of the SCISF on 
system performance. Without the SCISF (i.e., using a filter 
that provides only impulse shortening), the signal-to-noise 
ratio (SNR) decreases significantly at the output of the 
Fourier transform (i.e., FFT or DFT) to less than about 
7 dB. This decrease is due, in large part, to noise 
bleeding caused by the sidelobes of the Fourier transform. 
By contrast, with the SCISF, the SNR at the output of the 
Fourier transform tracks the SNR at the output of the SCISF 
within a few dB. Overall, the SCISF provides an improvement 
in SNR. 

The target spectral response of the SCISF, G d (w) and 
the actual frequency response, G(w), are derived from an 
energy constraint for the SCISF, i.e., the target spectral 
response must localize the energy of the effective impulse 
response within a desired frequency band. The energy 
constraint is combined with a desired spectral response 
based on the measured noise power spectral density. The 
resulting cost function (or error function) is then 
minimized to obtain a practical filter characteristic for 
the SCISF. This process is presented in detail below. 



- 14 - 



In the following derivation, all vectors are column vectors 
by default. Vectors are denoted by bold lower case letters (e.g., 
t) . The size m, of a vector t, is written t( w ,) . The components 
of a vector are denoted by lower case letters, e.g., 

The convolution of length mg + mh -i of vectors t^) 
and h(m h ) is denoted t*h. The Toeplitz matrix of a vector 

*M>Hm:n)4[trn- %f , is written T £ :w) , and T^x^^^x^. The 
Discrete Time Fourier Transform (DTFT) of a vector t may be 
represented as: 

ntt -1 

T(*)= I t n e' Jam . (!) 
rr=0 

The EDIR of the channel and the impulse response of the 
SCISF are expressed as h(m h ) and g{m 9 ) , respectively. The cyclic 
prefix has a length of m c per symbol. The energy of the 
effective impulse response must be localized to a contiguous 
region of target length m lt where m 1 <m ct while satisfying a 
spectral constraint. The energy criterion to be satisfied by the 
SCISF may be written as: 

I 2 

>a (2) 



.to /(m:m+jn,-l) 



for some 0<,m<m g + m h -2 and some 0<o<l. Defining the set 5 m as 



s = ge 



i(c*h) I 2 

Rm g ||Vto —/(m:m+m,-\)\ 

' iiT ^2 — ^ 



(3) 



fe'bf 

the impulse response of the SCISF must belong to S m for some m. 

Let (o u ...,w N be the location of the sub-carriers in the 
frequency domain. The spectral constraint may be applied by 
selecting g€S m , so that the cost function 
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tfaad-GMjf (4) 

i-l 

is minimized for the desired target spectral response G d {a>) . 

Typically, this optimization would have to be repeated for 
all possible m to select the filter impulse response, g, that 
achieves the lowest possible value of J. However, as discussed 
below, optimization may be limited to a few well chosen values of 
m. 

The determination of the filter impulse response may be done 
in two stages. First, the desired magnitude frequency response 
G d (o>) of the impulse shortening filter g over the bins used by 

the DMT system is obtained. Second, J is optimized over S m for 
a specific value of m. 

To determine the target spectral response G d (o>) , it is 
necessary to use expressions for the signal to noise ratios 
observed in the various frequency bins at the output of the DFT 
in the receiver. The DMT system has M tones (subcarriers) , N of 
which (those from M x through M 2 ) are used, and the communication 
channel has an analog frequency response H c (f) . The analog noise 
power spectral density observed at the input of the receiver A/D 
is s n (j) . Prior to conversion in the A/D converter, the received 
analog signal may be filtered by an anti-aliasing filter with 
transfer function H Q {f) . The EDIR in the absence of the impulse 
shortening filter is h{n) . After the A/D converter, the signal 
is fed into the impulse shortening filter with an impulse 
response of g(n) . The impulse shortening filter ensures that 
(h(n)*g(n)) is shorter in length than the cyclic prefix. G(eo) 
is the discrete time Fourier transform of g(n) . Under these 
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conditions, the expected signal energy M (k) observed in bin k at 
the output of the length-2AT receiver DFT is given by: 

where C, is a constant, 1/T the sampling frequency and % the 
transmitted power in bin k. The noise power rj(k) in bin k is: 



J L - 2 t)J 



(6) 



(0= 



M 



where C 2 is another constant and * denotes the convolution. 
Assuming that the noise in the bands corresponding to unused 
tones is sufficiently attenuated by the anti-alias filter, rj(k) 
is approximately equal to: 



(7) 



where a(n) is defined as 



W= J I 2/./ VvV *' 



(8) 



x = 



(9) 



M r -M 2 are the used tones and C 3 another constant. Defining x to 
be the vector of frequency magnitudes to be solved for as: 

the SNR in bin k can be seen to be of the form ^ , in which 

a k x 

are scalars and * k are vectors. 

To determine the target spectral response, x is chosen to 
maximize the bit throughput. Approximating the capacity of bin k 
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by VSNR(*) , the optimal spectral profile is obtained by 
minimizing the cost function F, where: 



_ f r ' (10) 
This minimization is performed over: 

^ = { X €R A ':|| X || = l, x .>Q y \$iz N } 
and can be accomplished by any one of a variety of constrained 
optimization strategies, as discussed in Bertsekas, Nonlin^r- 
Programming , Athena Scientific, Belmont, MA, 1995. A median 
filter may be applied to the output of the optimization algorithm 
to smooth the resulting target spectral response. 

Let A = H£ mj+jnj _ 2) and B= Hj^.,, . The energy constraint in 
equation (3) can be written as: 

Matrix A has full column rank, since it corresponds to a 
full convolution, so R M JA r A is invertible. Let R B jB r B 
Defining q = R^h , where R 0 / is the square root of the positive 
definite matrix R^, the energy constraint set can be written in 
terms of q as : 



„ J m r q r R^ 5 R s R^ 5 q | 
jq 6J! « • A / « q > g | . 

The next step is to reduce the dimensionality, i.e., the 
number of variables to search over in the optimization process. 
For example, in a video digital subscriber line (VDSL) applica- 
tion, an impulse shortening filter having a few hundred taps may 
be required. Searching_ over a variable space so large is 
difficult and impractical. Instead, the optimization is 



(13) 
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performed by searching over a cleverly chosen lower-dimensional 
subset of the variable space. This simplification in the 
optimization process may be done without significant reduction in 
the performance of the communication system. 

The reduction in dimensionality is accomplished by a 
transformation of variables. Let Cjr/ 5 r,r/ 5 . Let s = UzU r be 
the singular-value decomposition of C, where I is a diagonal 
matrix whose (i, i)th element is ff , and where the are arranged 
in descending order. If <r, <a, there is no feasible solution 
corresponding to delay m. If <r,ka, let 

u = [u,u 2 } i- Lo « hi . (14) 

where U, has size (m g ,m d ) and I, has size (m d ,n, d ) for some m d . 
These equations define the dimension of the range space of the 
matrix U, , over which the search is confined. The dimension m, 

a 

may be chosen either to include all a, greater than some 
threshold p,p<a, or m d may be a fixed number. Simulations 
indicate that md can be less than m g by more than an order of 
magnitude without significantly affecting communication system 
performance . 

The dimensionality reduction is achieved by a further 
transformation of variables: q = U,v . The energy constraint set 



now becomes 

V ~ f £ R ^ '■ T^f " "} = f 6 R ^ : v * °' a| * " v T h v ^ 0 } (15) 



using the identity U, r CU, = s, . 

Next, the cost function is expressed in terms of v. For a 

particular v, the corresponding g is given by g = R-/ 5 U lV , which 
leads to: 
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= Fg; F- : 
L 



I 

J 



(16) 



Let DjFAR^ 5 U, . Let D* a real (D) and D / jimag(Z)). D*andD 7 are real 
matrices of size (N,m d ). Let & T R „,\<,n<,N and dj*„, 1 <; n z AT be the rows 

R 1 

of D andD , respectively. Then: 

= ^ r («WiE, + d /( „d[„ ) = f\y 
where r n Jd R/W d^ +d ;//I d ; r /n . These definitions result in: 



(17) 



The gradient and Hessian of J 0 are: 



7 ;» = I 



^2 l--^£s£|: 



Hj 0 (v)=2,2 h--7== r n + ^ o r w w r ' 



(18) 



(19) 



The projection P v (y) of any yeR m d on to V is defined to be 



P v (y)= argmin||y-v|| 



(20) 



To develop an algorithm to optimize the cost function J 

0 

over V, an expression must be derived for P v (y). There is no 
closed- form expression for this projection operator, however 
there is a very efficient algorithm to compute it. If 
y ev, />(y) = y . If not, the projection onto V is the same as the 
projection on to its boundary, defined by: 

boundary (V) A {v € R m " :v * 0, dMI 2 - v r i,v = 0 } (21) 



The latter may be accomplished using LaGrange multipliers as 
follows. Defining a modified cost function S as: 

S(y)M\y - vlf +//(alMI 2 -v r i,v) (22) 

The equation Vy(v)=0 must be solved on the boundary of V. This 
reduces to the following simultaneous equations: 

[(l+/ia)I-/i2 1 ]v = 0 (23) 

a||v|| 2 -v r 2 1 v = 0 (24) 
Solving for v in terms of fu from (23) and substituting in (24) , 
the following equation is obtained for fii 



( \ 
y diag — ' 



y=0 (25) 



Equation (25) may be written as a polynomial equation of order 
2m d in pi. The polynomial equation must be solved for the real 
and positive roots, which may be done using one of the variety of 
efficient root-finding algorithms in the literature. One of the 
real, positive roots must be chosen by enumeration so that, when 
substituted into the expression for v, it leads to the smallest 

2 



y-v| 

Since the gradient and Hessian of the cost function J 0 and 
the projection operator onto V are available, the optimization 
can be done very efficiently. For example, the penalty method 
may be used, as described in Bertsekas, Nonlinear Programming . 
Athena Scientific, Belmont, MA, 1995, or an iterative strategy 
consisting of gradient descent followed by projection onto V. 
Both techniques have been tested and perform well. 

Other embodiments are within the scope of the following 
claims . 

What is claimed is: 
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